Comparison of scheduling methods for the learning rate of neural network language models (Modèles de langue neuronaux: une comparaison de plusieurs stratégies d'apprentissage) [in French]

If neural networks play an increasingly important role in natural language processing, training issues still hinder their dissemination in the community. This paper studies different learning strategies for neural language models (including two new strategies), focusing on the adaptation of the learning rate. Experimental results show the impact of the design of such strategy. Moreover, provided the choice of an appropriate training regime, it is possible to efficiently learn language models that achieves state of the art results in machine translation with a lower training time and a reduced impact of hyper-parameters. Mots-clés : Réseaux de neurones, modèles de langue n-gramme, traduction automatique statistique.